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ABSTRACT 


A  control  problem  with  a  system  modeled  as  a  nondeterministic 
finite  state  machine  is  considered.  Several  agents  seek  to  optimize 
the  behavior  of  the  system  under  a  minimax  criterion,  with  each  agent 
having  different  information  about  the  system  state.  The  nondetermi- 
.nistic  model  of  uncertainty,  combined  with  the  minimax  criterion,  lead 
to  equivalence  relations  on  thi .past  input/output  histories  of  each 
agent  which  generate  rather  simple  sufficient  statistics  for  the 
optimal  control  laws.  This  sh(2ds  light  on  the  basic  nature  of  decentra¬ 
lized  control  and  permits  complete  solution  of  a  particular  class  of 
problems . 
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OPTIMAL  DECENTRALIZED  CONTROL  OF 
FINITE  NONDETERMINI STIC  SYSTEMS 


I.  INTRODUCTION 

It  is  almost  a  cliche  that  computational  resources  are  becoming 
increasingly  inexpensive.  That  this  is  not  true  of  communications 
resources,  particularly  over  long  distances,  is  a  prime  motivation  for 
the  development  of  design  methods  for  computationally  distributed  solutions 
to  large  scale  control  problems  [1],  The  basic  structure  of  such  a  problem 
is  that  a  system,  subject  to  random  influences,  is  to  be  controlled  by  a 
group  of  several  powerful  control  nodes,  each  of  which  has  different 
information  about  the  events  in  the  system  due  to  communication  constraints. 
The  control  algorithms  in  each  node  must  be  designed  to  produce  good 
system  behavior  when  they  act  in  concert  with  the  system. 

Although  problems  of  this  nature  have  been  considered  for  many  years 
[2,3],  progress  towards  a  general  solution  has  been  surprizingly  slow. 

Many  problems  with  very  special  structure  have  been  solved  [4  -  9] ,  but 
problems  with  a  more  general  structure  have  defied  tractible  analysis 
[10,11],  Some  understanding  of  the  difficulties  encountered  has  been  gained, 

but  there  is  no  class  or  problems  with  a  aeneral,  dvnamic,  decentralized 
structure  for  which  exact  solutions  can  be  found  and  used  to  motivate 
further  investigation. 

The  motivation  for  this  work  is  to  cure  that  deficiency.  An 
exceedingly  simple  class  of  decentralized  control  problems  is  posed,  and 
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solved  completely.  The  simplicity  stems  from  the  assumed  system  model 
and  performance  objective  -  a  simplicity  which  is  deliberately  introduced 
to  illuminate  the  deep  structure  of  decentralized  problems,  yet  which 
preserves  the  essential  features  of  more  complex  applications:  stochastic 
dynamics,  a  general  information  structure  with  contrained  or  no  communication , 
and  a  desire  to  achieve  performance  which  is  optimal  in  some  precise  sense. 

The  system  to  be  controlled  is  assumed  to  be  described  as  a  finite 
state,  nondeterministic  automaton  with  inputs  supplied  by  several  control 
agents.  Each  agent  receives  am  observation  at  each  discrete  time  step 
which  indicates  a  set  in  which  the  current  state  must  lie.  Each  observation, 
coupled  with  knowledge  of  the  system  structure,  can  lead  to  inferences  about  the 

past  system  behavior,  hence  about  the  past  observations  of  other  agents  and  thus 
prediction  of  other  agents'  decisions.  This  simultaneous  interleaving 
of  inference  by  the  agents,  as  each  deduces  the  potential  actions  of 
others,  and  the  deductions  of  others  about  itself,  etc.  leads  to  some  of 
the  complexity  of  the  analytical  process. 

The  notion  of  using  nondeterministic,  as  opposed  to  Markov,  models  is 
not  new.  They  are  introduced  in  a  centralized  context  in  [12-13]  ,  and  used  in 
a  decentralized  problems  in  [14]  .  Their  advantage  is  that  the  set 


theoretic  operations  which  describe  their  operation  are  extremely  simple 
in  structure,  and  offer  a  rich  opportunity  for  manipulation.  Their  dis¬ 
advantage  is  that  they  tend  to  lead  to  "conservative”  control  laws,  as 
decisions  are  made  to  avoid  highly  unlikely,  expensive  transitions  which 
a  classical  probabilistic  approach  would  discount. 

Based  on  this  model  structure,  an  approach  to  addressing  the  problem 
of  designing  the  optimal  (in  the  appropriate  worst-case  sense)  decision 


rules  has  been  developed.  The  approach  is  based  on  the  identification  of 
the  set  of  sufficient  statistics  for  each  agent  to  use  and  the  dynamic 
relations  between  them;  these  sufficient  statistics  are  no  more  than  the 
intertwined  deductions  of  the  agents  about  each  other  truncated  at  the  point 
where  they  are  no  longer  productive.  The  set  of  these  statistics  form  an 
extended  state  space  over  which  dynamic  programming  may  be  used  to  derive 
the  optimal  decision  rules. 

This  approach  is  the  major  contribution  of  this  work.  It  enables  the 
study  of  a  class  of  decentralized,  stochastic,  dynamic,  optimal  control 
problems  which  is  sufficiently  simple  that  exact  solutions  may  be  obtained. 
These  solutions  may  serve  to  suggest  heretofore  unsuspected  properties 
of  optimal  solutions  in  general.  However,  the  possibility  of  direct 
application  of  this  approach  to  engineering  problems  does  exist:  many 
protocols  for  distributed  resource  allocation  in  computer  networks 
(e.g.  data  base  management  or  asynchronous  channel  access)  can  be  formulated 
in  this  framework  (but  have  not  yet  been  addressed) . 

In  the  sequel,  notation  and  the  problem  formulation  are  introduced 
in  section  II.  The  complexity  introduced  by  just  the  minimax,  nondetermi- 
nistic  optimization  approach  in  a  static,  decentralized  setting  is  discussed 
in  section  III.  The  sufficient  statistic,  called  the  information  relation,  and 
its  properties  are  then  developed  and  these,  coupled  with  the  results  of 
section  III,  solve  the  decentralized  estimation  problem  [15]  in  this 
context.  Finally,  section  VII  solves  the  control  problem  -  and  quite  un¬ 
expectedly,  this  involves  only  a  rather  straightforward  extension  of  the 


estimation  solution. 


II.  NOTATION  AND  PROBLEM  FORMULATION 
A.  Notation 

This  work  will  use  set  valued  functions 

f:X  -+  2Y  (2 

to  model  nondeterministic  behavior,  i.e.  y  6  p  £(x)  indicates  that  any 
element  y  of  the  set  £  q  Y  may  arise  as  a  result  of  applying  £  to  the 
point  x.  The  extension  of  f  to  a  function  on  the  power  set  of  X 

f  :  2  X  -*-2  Y  (2 

— e 

where 

f  (x)  =  U  f(x)  (2 

— e  “  _  — 

xex 

will  not  be  distinguished  from  £  itself.  Abusing  notation  slightly,  the 
preimage  of  y  under  f  will  be  denoted  by 

x  =  f"1  (y)  =  {  xj  yef  (x)>  (2 

Abusing  notation  more  than  slightly,  1  will  not  be  distinguished  from 
its  extension  to 

f'1  (£)  =  VJ  f-1  (y) 

~e  y®z 

-  {*1  znf'1  (x)  *  0  }  (2 

Finally,  any  cost  functional 

J  :  X  +*  (2 

will  have  an  extension 

J  (x)  =  max  J (x)  (2 

e  —  „ 

xex 

which  will  also  not  be  distinguished  from  its  restriction,  J. 

Subscripts  will  indicate  the  agent  associated  with  each  variable. 
Superscripts  will  indicate  elements  of  a  Cartesian  product  set?  e.g. 


xfc  -  (x(i),...,x(t))e  xfc. 

B.  Dynamics 

The  system  to  be  controlled  will  have  a  finite  state  space  X  with 
N  elements  and  dynamics 

x(t+l)  e  f(x(t),  u^t),  u2(t))  (2.8) 

u1 (t)  e  U  ,  u2 (t)  e  U2  are  inputs  supplied  by  two  decision  agents  (extension 
to  several  agents  is  straightforward) .  The  sets  and  U2  will  generally 
be  considered  finite,  although  allowing  them  to  be  real  intervals  will  aid 
the  interpretation  of  some  examples.  The  state  and  its  dynamics  may  be 
taken  to  include  any  interagent  communication  mechanisms. 

Observations 

yAt)  e  hMxtt))  (2.9) 

are  available  to  each  agent  at  each  time  just  before  (t)  is  to  be  selected. 
These  may  include  outputs  of  cononuni cations  channels,  representing  messages 
introduced  by  other  agents  at  earlier  time  steps. 

The  initial  state  is  assumed  fixed,  known  to  all  agents  and  is  denoted 
x^.  An  imperfect  initial  inform  ition  state  can  be  modeled  by  letting 

JE  (x°  ,u1  ,u2)  be  independent  of  u1#u2  and  equal  to  what  would  otherwise  be  the 
set  of  initial  states. 

C.  Information 

Each  agent  is  assumed  to  have  perfect  recall  of  all  past  observations 
and  decisions.  The  information  sequence  1^  includes  these  via  the  dynamics1 

Ii  < 0)  =  (  )  (2.10) 

ii(t+l)  *  yi(t+l)  .ii(t) 

Since  each  agent  has  perfect  recall,  past  decisions  may  be  formally  omitted 
from  (2.10)  as  they  may  be  recursively  derived  from  previous  observations  only. 


1 


where  (  )  is  the  empty  sequence,  and  •  denotes  concatenation.  The 

decision  rule  by  which  u^(t)  is  selected  is  restricted  to  being  a  causal 
function  of  the  local  information  sequence 

Yi  ( *;  t)  :  li(t)->-  U.  (2.] 

T 

I\  represents  the  entire  sequence  of  decision  rules  for  agent  i. 

D.  Objective 
A  cost  function 

Js  X  xox  x  u2  -*■  »  (2.: 

expresses  the  penalty  incurred  if  a  pair  (u^,u2>  are  applied  as  input  when 
the  system  occupies  state  x.  (Note  that  when  J  is  an  indicator  function 
on  some  subset  of  X,  then  the  problem  becomes  one  of  maintaining  x(t)  in 
a  fixed  target  set  in  X,  as  treated  in  [14]). 

T  T 

Taken  together,  the  dynamics  (2.8  -  2.10)  and  control  laws 
recursively  define  a  set  of  possible  joint  state,  control,  and  information 
trajectories.  Define 

J(lT,  r?)  =  max  max  J (x (t) ,u  (t) ,u  (t) )  (2. 

te{0,...,T}  x ( t) 
u^t) 

u2<t) 

where  the  x(t),  u^(t),  u2(t)  range  over  all  values  jointly  possible  at 
time  t  as  determined  by  this  recursion.  The  overall  objective  is  to 

minimize  J(f^,  T^)  (2. 


subject  to  (2.11) . 


III.  THE  STATIC  CASE 


Consider,  as  a  start,  the  one-step  decision  case  where  x°  gives  rise 

to  x  e  f,(x®) ,  each  of  two  agents  obtains  an  observation  y.  6  h.  (x)  ,  and 

1  — 1 

they  seek  to  select  Y.  :  y.  •+■  u.  to 

ill 

minimize 

V  Y2 


max 

x 

y. 


J(x,u  ,u2> 


A.  Necessary  Conditions 

The  person-by-person-optimality  (PBBO)  conditions  [4]  are  necessary 

conditions  which  the  optimal  decision  rules  and  y2  for  (3.1)  must 

satisfy.  They  are  established  as  follows. 

* 

Assume  y2  is  known.  Then  (3.1)  becomes 


minimize 

Y1 


max  J(x,u1,Y2(y2>)  ( 

x 

yl'y2 


Since  Yj  “ay  depend  on  y^,  it  is  sufficient  to  consider  the  values  of 
(x*y1»y2)  in  (3.2)  which  are  consistent  with  each  observed  value  of  y^ 

*  * 

Yltyl*  “  arg  min  max  J(x,ulfY2 (y2) )  ( 

U1  x  | 

y^  I  con  ;istent 

y2l 


The  values  of  x  consistent  with  an  observation  y^  (in  the  sense 
that  both  are  possible  simultaneously)  are 

x(y.)  ■  f(x°)  n  h”1  (y.)  ( 


-  k 
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The  x  defined  in  (3.4)  will  be  called  the  conditional  state  set,  as  it  plays 
the  same  role  in  nondeterministic  problems  as  does  the  conditional  state 
probability  distribution  in  Markov  problems.  The  values  of  y2  can  also  be 

A. 

inferred  to  lie  in  h2(x(y^)),  so  (3.3)  becomes 

*  A  *  A 
Yl*ylJ  “  arg  min  J(*(y1)»u1,  Y2  (h2  (x  (y^> ) )  (3.5) 

U1 

Symmetric  arguments  yield 

*  *  *  ^ 

Y2(y2)  =  arg  min  J (x(y2) (h^ (x (y2> ) ) ,u2)  (3.6) 

U2 

and  (3.5  -  3.6)  are  the  PBPO  conditions  for  the  static  problem. 

These  conditions  have  the  same  qualities  as  PBPO  conditions  for  other 
static  team  problems  [4,7,15].  First,  the  decision  made  in  response  to  y.^ 

depends  on  an  inference  of  the  other  agents'  decision,  hence  its  observation  - 

* 

the  so-called  "second  guessing"  phenomenon.  Second,  determination  of  y^ 

* 

for  a  specific  value  of  y^  depends  on  the  entire  structure  of  y2,  which  in 

* 

turn  depends  on  the  entire  function  Y1  •  Thus  these  conditions  do  not 

*  * 

allow  Y^  and  Y2  to  be  derived  on  a  point-by-point  basis  as  is  the  case  with  centra 
lized  problems;  a  joint  solution  for  the  both  complete  functions  must  be  found. 

These  characteristics  are  shared  with  classical  team  problems  [2,4,7]  •  this 
helps  substantiate  the  claim  that  th  .•  nondeterministic  formulation  is  qualitative 
similar  to  the  usual  Markov  one. 

B .  Example 

To  help  interpret  the  PBPO  conditions,  consider  a  specific  example. 

X  =  {l,2,3}  .  Agent  1  can  distinguish  1  from  {2,3}  ,  and  agent  2  may 

i  distinguish  2  from  f 1 , 3 }  ,  via  obseivations  (i.e.  h, (  •  ¥  h, ("’  -  h, (3)).  | 

,  i  1  1  | 


Decisions  are  to  be  drawn  from  U,  =  U  =  [0,1].  The  cos>.  function  is; 


.  ■»  M  J 


I 


b 
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J(x,u1#u2)  =  |  1-u^ 


I  u2”ul  I 


x  =  1 

x  =  2 

x  =  3 


(3.7) 


This  is  sketched  in  Figure  1  according  to  conventions  to  be  adhered  to 
throughout  this  work.  States  which  generate  the  same  observation  to  agent  1 
are  depicted  in  the  same  row;  those  with  the  same  image  under  h2  are  in  the  same 
column. J  is  plotted  as  a  function  of  u^  and  u^  for  each  state. 

It  is  immediately  apparent  that  Y^d)  and  Y2<2)  are  inconsequential 
(where  (l)  =  h^({l}),  {2}  =  h2(2}).  Thus  Y1  is  completely  characterized 
by  Yj^2),  and  Y 2  by  y2  (U  • 

Suppose  Y-^2)  =  8^-  Then  (3.6)  gives 


Y2(D  =  argmin  max  {u2,  |u2  -  B.J} 


(3.8) 


ft  8. 


(3.5)  gives 


Y.^2)  =  argmin  max  (l  -  u  ,  |32  -  uj} 


(3.9) 


These  are  not  the  usual  relations  encountered  in  optimization,  but  =  2/3, 

B2  =  1/3  can  be  found  as  the  unique  solution. 

This  example  demonstrates  the  character  of  decentralized  non- 
deterministic  static  optimization,  and  underscores  the  comments  at  the  end  of 
the  previous  section.  In  general,  these  conditions  are  quite  difficult  to 
solve  computationally;  it  is  critical  to  the  practical  solution  of  the  dynamic 
problem  that  the  domains  of  the  Y^  be  as  small  as  possible. 


a 
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C.  Structure 

There  is  some  simplification  possible.  Begin  with 

^  A  '  *  *  > 

Lemma  1:  If  x(y.)  =  x(y . ) ,  then  Y.  (v. )  =  Y. (v. ) . 

-  —  J  i  —  l  l  l 

Proof;  (3.5)  and  (3.6). 


This  states  that  the  conditional  state  set  is  a  sufficient  statistic  for 
the  static  team  problem,  as  is  the  conditional  state  distribution  in  Markov 
problems  [15] .  However,  the  set  theoretic  nature  of  the  problem  allows 
another  simplification: 


Lemma  2 : 


Proof; 


If  x(yj  CTx(y^)  ,  then  we  may  set 
*  A  *  • 

Vyi'  *  Vyi’ 

without  additional  cost. 

(3.5)  and  (3.6) ,  along  with  the  fact  that 

A  A'  A  A ' 

xCT*  ^=4  JU,u1.u2)  £  J  (x  ,ulfu2)  V  Ul,u2 


0 


This  has  no  analogy  in  Markov  problems.  It  is  a  result  of  the  worst 
case  objective,  since  if  21  is  a  worse  case  than  x,  in  terms  of  set  contain- 
ment,  then  there  is  no  harm  in  replacing  the  decision  corresponding  to  x 
with  that  of  x  . 

There  are  two  effects  of  Lemma  2.  The  first  is  beneficial  -  the 
complexity  of  Y.  may  be  reduced.  The  second  is  that  for  certain  events, 
namely,  if  y^  is  observed  rather  than  y^,  the  actual  cost  will  be  greater 
if  the  substitution  is  performed  than  if  it  were  not.  This  is  a  result 


of  the  fact  that  optimal  strategies  for  the  problem  as  posed  may  not  be 
unique,  and  that  consideration  of  events  other  than  the  worst  case  may 


% 


1 


M 


1 


i 


I 


4 


-12- 

allow  one  to  determine  that  one  strategy  is  intuitively  superior  to  another 
even  though  their  worst  case  performances  are  identical.  The  absence  of 
such  secondary  considerations  are  partially  responsible  for  the  conservatism 
associated  with  minimax,  nondeterministic  problems;  we  will  be  sensitive  to 
this  issue  in  the  sequel,  but  not  dwell  on  it. 

D.  Interpretation 

The  static,  nondeterministic  decentralized  problem  brings  out  three 
points  concerning  more  complex  problems: 

1.  The  PBPO  conditions  are  not  particularly  helpful  analytically. 

The  fact  that  there  is  a  tight  dependence  between  all  points  of  the 
decision  rules  suggests  that  the  solution  will  remain  tractable  only  if 
the  number  of  points  for  which  each  Y^  must  be  specified  remains  small. 

2.  While  many  problems  (e.g.  estimation  with  no  feedback  of  controls 
to  dynamics)  may  be  reduced  to  a  sequence  of  static  problems,  this  usually 
leads  to  a  sufficient  statistic  of  high  dimension  in  decentralized  settings 

(e.g.  sets  of  state  trajectories  [15]).  In  view  of  1,  this  is  not  sufficiently 
simple  to  lead  to  computable  solutions. 

3.  There  are  available  two  mechanisms  for  reducing  information  sets, 
and  thus  the  complexity  of  Y^.  Lemma  1  is  the  usual  equivalence  between 
information  sets  based  on  the  property  of  state;  Lemma  2  exploits  the  set 
theoretic  structure  of  the  problem. 

The  next  section  will  introduce  a  construction  to  which  both  mechanisms 
can  apply,  and  develop  some  of  its  algebraic  properties.  Section  V  returns 
to  the  decentralized  estimation  problem,  showing  how  its  decomposition  to 


a  sequence  of  static  problems  can  take  advantage  of  the  new  structure. 
Section  VI  achieves  similar  results  for  the  general  control  problem. 
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IV.  THE  INFORMATION  RELATION 
A.  Definition 

Consider  the  system  (2.8  -  2.10).  In  this  and  the  next  section,  we  will 

be  concerned  only  with  the  autonomous  case,  where 

x(t+l)  e  £_(x(t) )  (4.1) 

Definition;  The  global  conditional  state  set  at  time  t,  denoted  x^(t)CI  X, 
is  the  set  of  all  possible  states  which  the  system  may  occupy  at  time  t 
and  which  may  be  reached  along  a  trajectory  xfc  which  is  possible  given 
both  information  sequences  I^(t)  and  I^(t) .  & 

This  global  conditional  state  set  is  analogous  to  the  Markov  conditional 
state  distribution;  it  may  be  computed  recursively. 


Lemma  3;  The  global  conditional  state  set  may  be  computed  from 
x  (0)  =  (x°  } 

(4.2) 

x(t+l)  =  f(x(t))n  h"1  (y  (t+l)>n  h“1(y2(t+l)) 

Proof:  Set  manipulations . (4. 2)  first  generates  all  x(t+l)  which  may  arise 

from  states  in  x(t) ,  then  removes  those  which  could  not  generate 
the  observed  values  of  y^(t+l)  and  y2(t+2). 

O 

In  the  autonomous  case,  one  may  also  generate  local  conditional  state 
sets  x^(t)  based  on  I^(t)  alone; 

^(0)  =  {x°> 

*  -1  (4-3> 

X^t+I)  =f(xi(t))nhi  (yi(t+D) 

and  the  various  conditional  state  sets  are  related  by 


Corollary: 


(4.4) 


x ( t)  C  x  (t)H  x  (t) 
-  “1  “2 


Proof: 


If 


then 


By  induction.  x(0)  =  ^(0)0  x^  (°)  =  fx) 


x(t)  C  x  (t)  H  x2  (t) 


f(x(t))£  f  (x  (t)  H  x2(t)) 


C  f (x  (t))  H  f <x2(t)) 


(4.5) 


(4.6) 


(f(x(t))H  h^1(y1(t+l))  H  h"1  (y2  (t+1) )  ]  t 


(4.7) 


[l(*x(tJ)n  h"1  (yx  (t+1) ) )  n  [f(x2(t))n  h21(y2(t+l))] 

□ 


Note  that  the  second  containment  in  (4.6)  may  not  be  replaced  with 
equality  in  general;  figure  2  shows  an  example  where  (4.4)  must  be  a  contain¬ 
ment.  Thus  there  is  a  more  subtle  relationship  between  local  information 
sequences  and  the  global  state  set  than  that  provided  by  the  local  state 
sets. 


Definition:  An  information  relation  R  for  a  two  agent  problem  is  a  function 

from  two  sets  Z^(t)  and  Z2(t)  to  the  Dower  set  of  another  set:  X. 

For  autonomous  systems,  R  is  a  primitive  information  relation  if 
(t)  =  Y*'  Z2 ( t)  =  y2  '  X  is  the  state  space,  and 

R(yx*  =  2L(t)  q  (4.8) 

This  definition  will  be  extended  to  the  case  of  feedback  control  in 
section  VI.  For  autonomous  systems,  the  primitive  information  relation 


contains  all  of  the  logical  relationships  between  global  state  sets  which 
are  necessary  for  solving  decision  problems. 


» 


f(x°)  *{(,2,3,4} 
a)  Dynamics 


state 


{1,2} 

{1,2} 

{«.*} 

{2} 

{*} 

{1,2} 

M 

{2} 

b)  Observations 


1 

1 

{1.2} 

{1,3} 

W 

2 

2 

{I.2.3.4} 

{1.2.3.4} 

{1.2,3} 

c)  Sample  path 


Figure  2:  Example  of  Relation  Between  Conditional 
State  Sets 
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Before  developing  the  general  properties  of  information  relations, consider 
a  specific  example  of  a  primitive  information  relation, taken  from  system 
(a)  of  figure  3.  For  simplicity,  all  exampJes  in  figure  3  will  be  assumed  to 
have  an  observation  structure  whereby  agent  1  can  observe  the  row  of  the  diagram 
which  is  occupied  by  the  state,  and  agent  2  the  column.  (Thus  observations 
are  deterministic  functions  of  the  state) .  All  examples  have  x°  =  1. 

At  t  *  0,  I  =  I  =  (  ),  and  x°  =  1;  the  information  relation  is  as 
shown  in  figure  4a.  At  time  1,  Y*  =  Y*  =  *  and  the 

primitive  relation  is  shown  in  figure  4b.  Note  that  there  is  no  x 
consistent  with  observing  y^(l)  *  y2(l)  3  2,  as  only  state  4  could  cause 
this  and  it  cannot  be  reached  from  state  1  in  one  step.  This  is  indicated 
by  <f>  here;  subsequently  inconsistent  cases  are  left  blank.  In  every  other 
case,  the  global  conditional  state  set  is  a  singleton  as  the  system  is 
perfectly  observable  globally  (as  is  the  case  with  all  of  these  examples) . 

The  information  relation  captures  the  second  guessing  structure  -  the 
deductions  each  agent  can  make  about  others.  The  R  depicted  in  figure  4b 
summarizes  the  fact  that  agent  1,  receiving  y^d)  =  2,  knows  the  state 
is  3  and  that  agent  2  observed  y2(l)  =  1.  However,  in  this  case  agent  2 
cannot  distinguish  state  1  from  3  and  thus  cannot  determine  agent  l's 
observation;  it  can  conclude  that  if  agent  1  also  observed  1,  agent  1  in 
turn  could  not  distinguish  between  states  1  and  2.  This  logic  becomes  quite 
byzantine  even  for  this  simple  case?  the  information  relation  captures 
it  succinctly. 

At  time  2,  the  primitive  information  relation  is  shown  in  figure  4c. 

The  dashed  lines  indicate  that  it  is  comprised  of  four  components ,  each 


the  primitive  information  relation  which  would  have  resulted  had  the  system 


column 

1 


column 

2 


a)  Simple  Reducing 


c )  Infinite 


b)  Joint  Reducing 


Figure  3.  Example  Systems 
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started  at  the  prior  step  in  each  possible  state.  Thus  just  as  x°  =  1 
expanded  to  R(l) ,  so  the  corner  of  R(l)  corresponding  to  x(l)  =  1  expands 
to  the  same  structure  within  R(2).  The  information  relations  thus  preserve 
a  nesting  structure  generated  by  the  nes  :ing  of  the  observation  sets  (in 

the  sense  that  y^(l)  is  nested  in  (y^(2),  y^(l))). 

2 

If  the  order  of  the  elements  of  Y^  are  rearranged,  R(2)  can  be 
"unfolded"  and  rewritten  as  the  alternate  form  in  figure  4c.  Labels  for  the 
rows  and  columns,  as  well  as  impossible  cases,  have  been  dropped  for  clarity. 
This  alternate  tableau  form  clarifies  the  second-guessing  inference  structure. 

Finally,  figure  4d  shows  R(3)  as  generated  from  the  alternate  represen¬ 
tation  in  4c  by  replacing  each  cell  occupied  by  a  state  with  the  primitive 
information  relation  generated  by  starting  in  that  state.  An  alternate, 
unfolded  form  is  included  on  the  right. 

Figure  4d  suggests  the  utility  of  the  information  relation.  Agent  1 
can  receive  three  distinct  observation  sequences  from  which  it  may  infer 
that  x(3)  »  4.  For  one  of  these,  agent  2  may  not  be  able  to  distinguish 
whether  the  true  state  is  2  or  4;  for  the  other  two,  agent  2  also  knows 
that  x  =  4 .  In  fact,  the  two  cells  at  the  upper  right  of 
the  tableau  are  completely  isomorphic  in  their  relations  to  the  rest 
of  the  structure,  and  it  will  be  shown  that  the  optimal  decisions  for 
agent  1^  to  make  in  these  two  cases  are  identical.  Thus  agent  1  need  not 
distinguish  between  the  two  observation  sequences  leading  to  these  tyo 
identical  rows  in  the  tableau;  the  information  set  may  be  collapesed 
to  a  smaller  sufficient  statistic  by  identifying  the  two  cases. 

Thus  the  motivation  for  studying  the  information  relations  is  to 
reduce  a  primitive  relation  to  one  of  smaller  dimension  which  still  serves 
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for  the  generation  of  optimal  decision  rules.  The  generic  sets  Z^  wiich 
comprise  any  relation  will  be  aggregations  of  the  primitive  information 
sets  I^(t).  The  remainder  of  this  section  establishes  the  algebraic 
structure  of  information  relations. 


B.  Homomorphisms . 

The  global  conditional  state  sets  have  a  lattice  structure  superimposed 
on  them  by  the  set  containment  relation.  Information  relations  have  a 
similar  structure  which,  while  not  a  lattice,  is  (almost)  a  partial  order. 

Definition:  A  homomorphism  <J>  from  R,  defined  from  Z^  and  to  2  ,  to  R' 

defined  from  Z,,  Z*  ,  to  2  ,  is  a  pair  of  functions  <f>  = 

z  •  * 


* 


l 


„  2L 
2:  2 


I 


(4.9) 


satisfying 


R(z1#z2)C  r'  (*j_)  ,<j>2  (z^  ) 


v  z2  e  z2 


(4.10) 


r(£1,z2)9  r‘  (♦(*1)  ,4>2(z2))  vl1ez1 

O 

In  the  tableau  representation,  a  homomorphism  from  R  to  R'  is  an  association 
of  the  rows  and  columns  of  R  to  those  of  R'  such  that  the  state  set  in 
each  cell  of  R  is  contained  in  the  state  set  of  the  corresponding  cell  in 
the  image. 


Definition;  An  information  relation  R  is  contained  in  another  R'  (R  £  R' ) 
if  there  is  a  homomorphism  from  R  to  R' . 


This  containment  relation  is  a  generalization  of  set  containment 

When  Z_  =  Zl  =  {<p},  and  R(z  ,z  )  is  at  most  a  singleton,  this  is  exactly 

Z  Z  1  2 


set  containment  where  Z^  and  Z ^  index  elements  of  subsets  of  X,  (e.g. 
figure  5a).  When  may  be  any  set,  this  defines  containment  between 

two  classes  of  subsets  of  X  (figure  5b) :  note  that  one  class  may  be 
contained  in  another  although  the  second  has  fewer  elements.  Containment 
between  information  relations  may  be  neither  antisymmetric  nor  related  to 
the  size  of  the  Z ^  and  Z|  (figure  5c). 

The  containment  relation  is  clearly  reflexive  (if  4> .  is  the  identity  map) 
and  transitive  (through  composition  of  4>^'s).  It  thus  imposes  a  structure 
on  the  set  of  information  relations  which  is  a  partial  order  on  equivalence 
classes,  where: 

Definition:  R  is  equivalent  to  r'  (r^r*) 
if  R  Cr*  and  R  2  R*  .  q 

Both  the  partial  order  and  equivalence  relation  have  useful  interpretations 
in  the  decentralized  problem. 

C.  Automorphisms 

This  section  focusses  on  the  equivalence  Son  information  relations. 

Definition:  An  automorphism  <J>  on  an  information  relation  R  is  a  homomorphism 

from  R  to  R.  If  both  ♦1  <f>2  are  1:1,  then  is  311 

isomorphism.  Otherwise,  it  is  a  reducing  automorphism . 

a 

Clearly  isomorphisms  on  an  information  relation  correspond  to  permuting 
rows  and  columns  of  a  tableai ,  and  are  of  little  conceptual  interesbi  There 
are  automorphisms  which  are  not  isomorphisms,  and  these  are  of  considerable 
interest. 


Definition:  An  information  relation  R  is  irreducible  if  all  automorphisms 

on  R  are  isomorphisms.  Otherwise  it  is  reducible. 


n  13)5)  g  11121315171 
a)  Set  containment 


As  the  terminology  suggests,  a  reducing  automorphism  will  be  used 
to  simplify,  in  some  appropriate  sense,  an  information  relation.  Provided 
we  can  show  that  using  the  simplified  information  relation  in  decision  rule 
design  yields  results  no  worse  than  the  original,  then  there  will  exist  a 
mechanism  for  generating  sufficient  statistics  which  are  simpler  them  the 
I^(t).  The  reducing  automorphism  provides  this  mechamism. 

X 

Definition;  The  reduction  of  an  information  relation  R  :  Z^  X  Z 

by  a  reducing  automorphism  <{>  =  (<f>i»<f>2)  is  an  information 
relation  R'  :  x  <|>2(Z2)  2  where 

R'(<^1(z1),  <j>2(z2))  =  ROfr^z^,  ♦2(z2))  (4. 

□ 

For  example,  figure  6a  shows  successive  reductions  of  the  primitive 
information  relation  at  time  3  for  the  system  in  figure  3a  (as  was  derived 

in  figure  4)  by  the  reducing  automorphisms  indicated  by  arrows.  The  result 

is  an  irreducible  relation.  In  this  case,  the  irreducible  relation  can 

be  constructed  by  a  sequence  of  automorphisms  which  only  reduce  Z^  or  Z2 

alone.  One  might  conjecture  that  this  is  always  the  case,  but  figure 

6b  shows  how  the  primitive  information  relation  for  system  3b  requires  a 

joint  reduction  of  both  and  Z2  at  the  second  step  in  order  to  produce 

an  irreducible  relation. 

However,  the  fact  that  every  information  relation  cam  be  reduced  to 
an  irreducible  one  by  suitable  compositions  of  reducing  automorphisms 
(thus  by  some  single  reducing  automorpl'ism)  is  important. 

D.  The  Core 

This  construct  is  the  most  essential  part  of  this  work. 


Definition;  A  core  of  an  information  relation  R, denoted  core  (R)  or  R*,is 


an  irreducible  information  relation  obtained  from  R  by  some 
automorphism.  O 

The  core  has  a  number  of  interesting  properties.  The  most  basic  rely 
on  the  following  lemma. 

Lemma  4;  If  two  irreducible  ir formation  relations  R  and  R'  are  equivalent 
under  then  they  are  equal 


Proof :  If  rSR',  then  there  is  a  homomorphism  <J>  from  R  to  R' ,  and 

another,  <J>'  ,  from  R'  to  R. 

Consider  the  composed  homomorphism  <j>  from  R  to  R; 

♦  jCZjl  -  (*1(21))  (4'l2) 

That  is  indeed  a  homomorphism  from  R  to  R  is  shown  by 

R(zlfz2)C  R-  (4>1(z1),  4>2  <z2) )  V  z2  6  Z2  (4.13) 


since  <t>  is  a  homomorphism  from  R  to  R' ,  and  in  turn 

R'  ),t2<z2))  £  r(4>^(4>1(z1),  $2($2(z2)))  v  z2  e  z2 

(4.14) 

by<J>'beinq  a  homomorphism.  Thus  <J>  is  an  automorphism  on  R,  and  in 
fact  must  be  an  isomorphism  since  R  is  irreducible.  However,  the 
composition  of  4>  and  <t>'  can  be  an  isomorphism  if  and  only  if  <{>  and4>' 
are  isomorphisms;  hence  R  and  R'  are  equal. 

Up  to  isomorphism.  This  qualifier  will  be  left  implicit  in  the 

sequel. 


This  lemma  immediately  gives: 


Theorem  1:  The  core  of  an  information  relation  is  unique.  Moreover,  if 
R  R’ ,  then  core  (R)  =  core  (R* ) . 

Proof:  Any  core  (R)^  R,  since  a  homomorphism  from  R  to  a  core  (R) 

exists  by  definition,  and  a  homomorphism  from  core  (R)  to  R  exists 
by  construction:  core  (R)  is  a  (perhaps  relabeled)  restriction 
of  R  to  subsets  of  Z^  and  Z ;  construct  the  homomorphism  from 
elements  of  these  subsets  back  into  their  original  values  in  Z^  and  Z 
If  two  cores,  core  (R)  and  core' (R)  exist,  then 

core  (R)S  RScore'  (r)  (4.15) 

Both  are  irreducible;  ly  transitivity  of  and  lemma  4  they  must 
be  equal,  hence  unique. 

Moreover, 

core  (R)S>  Rv^R'£=>  core  (R')  (4.16) 

similarly  implies  core  (R)  =  core  (R'K  g 

Thus  all  information  relations  in  an  equivalence  class  of  SS  share  the 
same  core.  Restricted  to  irreducible  information  relations,  the  containment 
relation  C  is  antisymmetric  {R?r  R'  and  R  2  R'  R  =  R’ ) ,  transitive,  and 
reflexive,  thus  inducing  a  partial  order  on  the  irreducible  information 
patterns.  These  are  the  two  important  structural  properties  of  information 
r  lations  for  any  time  t. 

E.  Dynamics 

Section  A  informally  showed  how  primitive  information  relations  for 
the  same  autonomous  system  at  different  times  related  to  one  another. 

Section  D  derived  a  way  to  reduce  promitive  relations  to  their  core,  thereby 
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creating  simpler  relations.  This  section  formalizes  the  process  by  which 

any  relation,  primitive  or  reduced,  can  be  extended  over  time  and  presents 

the  basic  property  inherent  in  that  process. 

At  any  point  in  time  Z.(t)  in  R(t)  will  represent  a  reduced,  perhaps 
t 

trivially,  version  of  •  There  is  a  natural  way  to  extend  this  aqqreqation 
to  its  counterpart  at  time  t+1. 


Definition;  The  expansion  of  an  information  relation  R(t)  to  another 
relation  R(t+1)  is  denoted 

R(t+1)  =  F (R (t) )  (4.17) 

and  is  constructed  by  setting 

Zi(t+1)  =  Zi(t)  x  Y  (4.18) 

and 

R(t+1)[  zL(t+l) ,  ::2(t+l)] 

=  R(t+1)  [ (zx (t) ,yx) ,  (z2(t),y2)l  (4.19) 

=  f  (R(t)[Z;L(t),z2(t)])n  h"1  (Y1>  D  h”1  (y2}  ° 


The  structure  of  this  expansion  is  captured  in 


Lemma  5: 


If  R (t)  is  the  primitive  relation  at  time  t,  then  F (R(t) )  is 
the  primitive  relation  for  time  t+1. 

U 


Proof ;  By  definition,  R(t)  :  Y^  x  Y2  -*■  2X,  and  R(t)  (y^,y2>  =  jc(t)  , 

the  global  conditional  state  set  based  on  (y^,  y2) .  Identifying 
y^  in  (4.19)  with  the  observation  y^(t+l),  yields 


R(t+1)  (y*+1,  y2+1)  =  f(x(t))  n  h^ly^t+l))  O  h^tytt+l)) 

=  x (t+1 )  (4.20) 


by  (4.2).  Thus  R(t+1)  is  a  primitive  information  relation. 

□ 
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In  general,  the  expansion  process  augments  the  Z^(t)  with  y^(t+l) 
and  computes  the  conditional  state  set  for  z^(t),  (t) ,  y^(t),  and  y2(t) 

based  on  the  system  dynamics  f_  and  the  representation  in  R(t)  of  a 
conditional  state  set  for  z^(t)  and  (t) .  Section  B  introduced  containment 
between  information  relations  to  convey  the  notion  that  one  was  related 
to  a  substructure  of  another.  Expansion  preserves  containment. 


Lemma  6:  If  RCR',  then 

F  (R)  <Z-  F(R'  ) 


Proof i  Since  ROR'»  there  exist  functions  4k:  -+  Z^  satisfying 

(4.10).  Construct  functions  4>*:  z-  x  Y.  "*■  Z !  x  Y. 

11111 

where 


<J>i(Zi,yi)  = 

Then  for  all  z and  y2» 

F(R((z1,y1),  <z2'y2>)) 

=  £(R(z1,z2))^')h11(y1)^h21(y2) 
=  f(R'(4»1(*1),  $2U2))vOh‘1 
by  (4.10)  and  set  inequalities,  and  this 
=  f(r')  (4>^(zIry1)  ,  ^(z2,y2) 


(4.21) 


(4.22) 


(4.23) 

(4.24) 


(4.25) 


This  establishes  the  first  half  of  (4.10)  for  4>+  ;  the  other  half 
is  shown  bv  a  symmetric  argument.  Thus  <f>+  is  a  homomorphism  from 
F  (R)  to  F(R');  hence  F(R)C  F(R'). 

0 


This  sets  up  the  second  major  result: 
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Theorem  2:  Let  R(t)  and  R{t+1)  be  primitive  information  relations  at 

successive  times.  Then 


core (R(t+1) )  =  core (F (core (R(t) ) )  (4.26) 

Proof:  By  Lennra  5, 

R(t+1)  =  F (R(t) )  (4.27) 

From  Theorem  1 

core  (R(t) )  ^>  R(t)  (4.28) 

Hence  applying  leirena  6  to  each  containment  in  5^. 

F  (core  (R(t) ) )  <^,F(R(t))  (4.29) 

Then  bv  Theorem  1  and  (4.27) 

core  (F (core (R(t) ) ) )  =  core  (R(t+1))  (4.30) 


O 

Thus  the  cores  of  the  primitive  information  relations  can  be  computed 
recursively,  rather  than  just  directly.  This  is  the  analog  of  the  fact 
that  conditional  state  probability  distributions  for  Markov  processes  may 
be  computed  recursively,  rather  than  requiring  reference  to  I(t)  at  each 
time,  and  has  the  same  structural  implications.  The  size  of  the  primitive 
information  relations  R(t)  generally  increases  exponentially  with  t, 
since  =  Y^.  However,  the  cores  of  the  primitive  information  relations, 

R* (t) ,  may  remain  much  more  manageable  in  size. 

Figure  7  shows  the  recursive  computation  of  the  core  information 
relations  for  the  example  of  figure  3b.  F  denotes  expansion;  *  denotes 
reduction  to  an  irreducible  relation.  First,  notice  that  the  reduction  of 
R(z)  led  to  an  expansion  of  R*(z)  which  was  a  bit  simpler  than  the  primitive 
R(3)  displayed  in  figure  6,  so  reductions  are  indeed  cumulative  over  time. 
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More  importantly,  notice  that  R*(3)  =  R*(2)  (isomorphically) ,  and  thus 
the  core  information  relation  for  this  system  is  fixed  for  all  future  time 
at  a  finite  size! 

Definition:  The  steady  state  core  information  relation  R*,  if  it 

exists,  is  the  core  of  some  primitive  information  relation 
R(t)  and  satisfies 

R*  =  core (F  (R*) )  (4.31) 

□ 

If  a  steady  state  core  can  be  found  for  a  system,  then  a  great  deal 
of  the  work  required  to  solve  the  system  is  complete.  However,  not  all 
systems  have  a  steady  state  core;  the  system  in  figure  3c  generates  cores 
of  the  form  shown  in  figure  8.  (In  fact,  these  are  identical  to  the 
primitive  information  relations) .  While  an  irreducible  information  relation 
can  be  found  for  this  system  whi:h  satisfies  (4.31),  and  which  contains 
(but  is  not  contained  in)  R*(t)  lor  all  time,  it  is  not  yet  clear  what  the 
relation  to  the  notion  of  a  core  for  an  infinite  horizon  problem  is. 

This  question  is  revisited  in  Appendix  A. 

A  final  property  of  the  core  dynamics  R*(t+1)  =  core  (F(R*(t))  is 
illustrated  by  the  system  of  figure  3d.  RM2)  is  shown  in  figure  9j 
notice  that  it  separates  into  two  pieces:  the  solitary  2  state  shares 
no  rows  or  column  with  any  other  entry.  What  is  the  physical  reason  for 
this? 

The  solitary  2  is  generated  by  a  trans  i.tion  from  4  to  2.  This  is 
an  unusual  transition  in  that  both  agents  observe  it  unambigously  -  for 
agent  1,  it  is  the  only  transition  from  row  2  to  row  1;  for  agent  2  it  is 
the  only  transition  within  column  2.  Thus  its  occurrence  leads  each  agent 
to  know  exactly  what  happened,  and  that  the  other  agent  knows  exactly  what 
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happened,  etc.  This  is  an  event  which  is  simultaneously  and  unambigously 
observable  to  both  agents,  and  thus  decouples  it  from  any  other  events 
and  second  guessing  logic. 

Definition:  An  information  relation  R  splits  if  the  sets  Z.  can  be 

partitioned  into  nonempty  disjoint  subsets  z\  Z^  with 

r(z^,  z ” )  =0  v  z1  e  z^,  V  z^e  z"2 

(4.32) 

"  1  II  II  •  • 

R(zi  ,  z2)  =  0  v  zx  e  z1  ,  v  z2  e  z2 

o 

If  a  relation  splits,  then  the  expansion  of  each  part  of  the  relation 
can  be  computed  separately  (i.e.  if  R  splits,  then  F(R)  splits  in  the 
compatible  way).  In  the  context  of  core  dynamics,  this  means  that  the 
two  pieces  evolve  separately.  This  suggests  a  representation  for  the 
core  dynamics  as  shown  in  figure  9b  -  R*(l)  expands  and  splits  into  two 
parts  -  one  of  which  reduces  back  to  R*(l),  the  other  of  which  evolves 
separately.  In  this  case,  however,  the  steady  state  core  as  still  that 
illustrated  in  figure  9a. 

F-  Local  Observers 

Thus  far,  attention  has  centered  on  off-line  calculations  -  the  reduction 
of  primitive  information  relations  to  core  relations,  and  the  core  dynamics. 
The  ostensible  purpose  of  this  is  to  aid  the  design  of  on-line  decision 
rules.  The  connection  between  the  two  is  the  dynamics  of  the  elements  of 
Z^(t)  which  are  implied  by  the  core  dynamics. 

The  core  dynamics  alternately  expand  Z^ft)  by  appending  , 
a  new  observation,  and  then  reduce  it  via  an  equivalence  relation 
implied  by  an  automorphism  <j>^.  This  equivalence  relation  combines 


-28- 


those  elements  of  Z^  x  Y^  which  need  not  be  distinguished  in  the  future  as 

far  as  the  core  dynamics  are  concerned.  The  entire  sequence  of  these 

equivalence  relations  map  every  observation  sequence  y^  into  some  element 

of  Z. (t)  ,  and  thus  dictate  the  structure  of  a  finite  state  machine  with  Y 
1  i 

as  an  input  set  and  Z^t)  as  the  states. 

Definition:  The  local  core  observer  for  agent  i  is  the  system  with 
state  set  Z^(t)  and  dynamics 


zi(t+l)  =  fi(zi(t),  yi(t+l)) 


=  <j>.(z.(t),  y.(t+l)) 


(4.33) 


where  the  zi  (t )  are  the  components  on  which  R  (t)  is  defined,  and  <Jk  is  a 

*  •* 

component  of  the  reducing  automorphism  used  to  reduce  F (R  (t))  to  R  (t+1) . 

it 


For  example,  consider  the  system  of  figure  10a.  Its  steady  state 
core  is  shown  in  figure  10b,  with  elements  of  Z 1  and  Z2  labeled  arbitrarily. 
Figure  10c  shows  the  expansion  F(R*) ,  with  and  4>2  as  required  to 
reduce  it  back  to  R*.  Figure  lOd  is  the  resulting  structure  for  the  local 
observer  of  agent  1  -  agent  2  has  an  isomorphic  structure  by  the  symmetry 
of  the  problem.  Arcs  are  labeled  with  the  values  of  y^  causing  transitions 
in  the  observer;  no  arc  labeled  1  leaves  as  this  represents  a  case  where 
agent  1  knows  x(t)  =  2,  and  2  is  followed  only  by  4  which  gives  the 
observation  2. 

The  dynamics  of  these  observers  are  intertwined  despite  the  fact  that 
they  operate  on  the  basis  of  different  observations.  This  fact  shows 
up  in  the  information  relation  on  which  they  are  based.  R*(z^,z2>  is 
the  subset  of  states  which  the  system  may  occupy  when  each  agent  i's 
observer  is  in  state  z^.  When  RMz^z^  =  0  ,  it  is  impossible  that 
and  z2  be  occupied  simultaneously  -  thus  the  observers  are  at  least 


partially  synchronized. 


Other  insight  as  to  tne  meaning  of  the  local  observer  states  can  be 
gained  from  noting  that  R* (z  ,Z  )  is  the  complete  set  of  states  which  the 
system  may  occupy  if  agent  l's  observer  is  in  state  z^.  It  is  the  union 
of  the  sets  of  states  which  the  system  could  occupy  given  each  y^  which 
leads  to  z^.  Thus  the  state  sets  corresponding  to  each  z^  are  related 
to  the  local  conditional  state  sets  defined  in  (4.3).  However,  the 
relation  is  rather  subtle,  as  both  z^2  and  z  indicate  that  x  e(3,4}  ; 
they  must  remain  distinct  due  to  the  existence  of  agent  2  with  different 
information. 


Thus  the  core  dynamics  define;  some  automata  for  processing  local 
observations  in  a  way  which  maintains  the  relationships  described  in  R* . 


G.  Special  Cases 

There  are  certain  nondeterministic  systems  which  have  more  structure 
than  the  general  case  discussed  thus  far.  For  example,  a  system  yields 
the  equivalent  of  centralized  information  if  y^ft)  =  y2 (t)  at  all  times. 
This  special  case  appears  in  R(t)  in  that  it  is  diagonal. 


R(t) (y^,y2> 


0 

A 

x(i.) 


if  Y, 


else 


(4. 


The  local  core  observers,  in  this  case,  are  perfectly  synchronized  and 
each  implements  a  reduced  order  realization  of  the  conditional  state  set 
computation  (4.2).  The  realization  is  j educed  order  as  any  reachable 
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conditional  state  set  which  is  contained  in  another  is  treated  as  equivalent 
to  that  other.  Thus  only  the  maximal  (in  the  sense  of  set  containment) 
reachable  conditional  state  sets  are  tracked. 

A  slightly  more  general  case  is  where? 

Definition:  The  observations  cf  agent  1  are  included  in  those  of  agent  2 
if  y^(t)  =  h21(y2(t)),  for  some  h21- 

J 

Thus  if  y2(t)  is  a  vector,  y^(t)  may  be  one  component  of  that  vector. 

In  this  case,  the  information  relations  have  the  property  that  for  any 
there  is  exactly  one  z  for  which  R(z^,z^)  f  0.  This  is  true  for  the  primitive 
information  relations  since  z2  =  y2  uniquely  determines  the  y^  that  will 
arise  •  This  property  is  preserved  wnen  R(t)  is  reduced  by  any  automorphism; 

proof  of  this  fact  is  deferred.  Figure  11  illustrates  such  an  information 
relation. 

In  this  case,  the  local  core  observer  for  agent  1  turns  out  to  just 
be  a  reduced  order  realization  of  the  local  conditional  state  set  computation. 
The  local  core  observer  for  agent  2  is  a  reduced  order  realization  of  the 
conditional  state  set  for  the  combined  system  and  local  core  observer  for 
agent  1.  This  is  the  structure  demonstrated  in  the  example  in  [15]. 

A  final  special  case  has  one  agent,  say  1,  operating  open  loop- 
without  observations.  In  this  case,  set 

y1(t)  =  0  =  h12(y2(t))  (4.35) 

to  see  that  it  is  a  variation  of  the  above  case.  However,  now  always 
has  exactly  one  element;  z^(t)  indicates  the  steady  state  conditional 
state  set  (generally  X  unless  part  of  the  state  space  is  not  reachable) , 

A 

and  z2<t)  realizes  a  reduced  order  computation  of  x(t). 


H.  Summar 


This  section  has  developed  the  notion  of  an  information  relation,  and 
some  of  the  properties  thereof.  The  properties  all  stem  from  the  definition 
of  information  relation  containment  via  homormorphisms  (4.10).  The  basic 
premise,  as  yet  unjustified,  has  been  that  this  definition  is  useful  in 
providing  the  structure  needed  to  solve  decentralized  nondeterministic 
decision  problems.  The  structure  resulting  from  this  definition,  particularly 
the  core,  is  now  known;  the  next  section  shows  what  it  can  do  in  a  subset 
of  the  problems  of  interest. 

V.  DECENTRALIZED  ESTIMATION 

This  section  continues  the  assumption  of  the  previous  one  that  the 
system  dynamics  are  autonomous,  but  introduces  the  objective  function  in 
order  to  derive  decision  rules.  Since  the  cost  function  J  compares  agents' 
decisions  with  the  true  state,  these  will  be  referred  to  as  decentralized 
estimation  problems.  They  can  be  solved  by  (a)  reducing  them  to  a  sequence 
of  static  problems  of  the  type  addressed  in  section  III,  (b)  relating  the 
static  problems  to  the  information  relation  and  its  structure,  and 
(c)  combining  the  resulting  decision  rules  with  the  observer  dynamics 
necessary  to  compute  sufficient  statistics  on-line.  Some  special  cases  will 
be  of  particular  interest. 

A.  Reduction  to  Static  Problems 

The  general  decentralized  estimation  problem  is  to  find  the 

min  max  max  J  (x  (t)  ,u.  (t)  ,u  (t) )  (5.1) 

„T  T  te(l,. . . ,T)  x(t)possible 
1  2 


relation  can  play 
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Definition:  The  optimal  value  of  an  information  relation  R  in  a  non- 

deterministic  decentralized  estimation  problem,  denoted  J* (R) , 
is 


min 


max 


VY2  ZieZl 

Z2eZ2 


J(x.y  (z^  ,Y2(z2)  ) 


(5.6) 


x6R(z1<z2) 


with  Z^  and  Z2  the  sets  on  which  R  is  defined,  and  the  restriction 


Y.  :  Z  .  ■+  U  . 
'  1  l  l 


(5.7) 


Thus  if  R  is  a  primitive  information  relation,  J* (R)  is  the  optimal 
value  of  (5.5),  since  Z =  and  R  specifies  the  set  of  x(t)  possible 

(perhaps  empty)  for  each  (y^,  y^) .  Naturally  we  are  interested  in  simpler 
information  relations;  the  following  lemma  establishes  the  connection  between 
information  relations  and  their  optimal  values. 


Lemma  8:  If  R  and  R'  are  information  relations,  then 

R  C  r'**JMR)  <  JMR')  (5.8) 

Proof ;  If  RC  R'  ,  then  a  homomorphism  <f>  =  (4>1»4>2)  exists  from  R  to  R* . 
Consider  any  strategy  y'  =  (y^,  y'2)  with 

Y'.  :  z'.  -*■  U.  (5.9) 

l  l  l 

Build  a  strateav  Y  =  with 

Y.  (z.)  =  y'  (4>  (z.)) 


(5.10) 
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J  (x,  Y1<z1>  »  Y?  (z2)) 


(5.11) 


xeR(z1#z2) 


max  J(x,y1('I>1(z1))*  Y2(‘t>2(z2))) 


xe  r(z1,z2) 


<  max 


j(x,y'1(z,1>»  y2(z2^ 


(5.12) 


xe  r' (z^,z p 


where  the  inequality  stems  from  the  fact  that 


4> .  (Z . )  C  Z ! 
li—i 


R(z  ,z2)  CR'l^lz^ili  2(Z2*) 


(5.13) 


frcan  the  definition  of  a  homomorDhism.  Since  (5.12)  holds  for  all  y* , 
evaluate  it  at  ay'  which  achieves  J*  (R‘) •  The  corresponding  y  defined 
in  (5.10),  must  achieve  a  value  of  (5.11)  no  larger  than  J*(R'), 
and  the  best  y  is  at  least  as  good  at.  this  one,  so  the  conclusion  holds 
This  immediately  suggests  the  third  major  result: 

Theorem  3:  Let  R(t)  be  a  primitive  information  relation  and  R* (t)  its  core. 


J*  (R  (t) )  =  J*  (R*  (t) ) 


(5.14) 


Proof :  From  theorem  1,  R(t)^R 


App lyin«)  Lemma  8  in  both  directions. 


the  conclusion  follows. 
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This  theorem  implies  that  the  pair  of  decision  rules  y^ ,  y  which 
achieve  the  optimum  value  of  (5.6)  when  R  is  a  primitive  relation  perform 
exactly  as  well  as  the  rules  optimizing  (5.6)  for  tne  core  of  that  relation. 
Since  the  former  are  functions  on  Y^ ,  they  are  more  cumbersome  than  the 
latter,  which  are  functions  of  only  the  of  the  core.  Since  each  agent 
can  compute  the  value  of  z^t)  representing  the  situation  given  an  observation 
sequence  y^  by  using  its  local  core  observer,  the  solution  to  (5.6) 
with  R*(t)  can  be  implemented  on  line.  Thus  the  complexity  of  the  problem 
can  be  significantly  reduced  from  that  of  (5.5). 

C.  General  Solution 

The  procedure  for  solving  the  nondeterministic  optimal  decentalized 
estimation  problem  is  thus: 

1.  Compute  the  core  dynamics  (4.18  -  4.19,  4.11) 


R*tt+1)  =  core  <F(R*(t))) 

RMO)  =  jR({zi),  {z°})  =  {x°}| 

2.  Derive  from  these  the  local  core  observers  (4.33) 

A 

zi(t+l)  =  fi(z.(t),  y^t+1)) 


(5.15) 


(5.16) 


Find  the  core  decision  rules  Y*  :  •*  (k  minimizing 


max  J(xx  y1(z1),  y2(z2)) 
Z1 


xe  R* (t)  (z^z  ) 


(5.17) 


4.  Implement  the  local  core  observers  and  decision  rules 


Ul(t)  =  Yi (zi (t) ] 


(5.18) 


Thus  a  recursive  solution  can  be  found  based  on  the  structure  of 
the  core  information  relation.  The  solution  of  (5.17)  can  be  obtained 
as  described  in  section  III  by  exploiting  the  PBPO  necessary  conditions 

★  it 

and  finding  the  best  pair  (y^,  y^)  which  satisfy  them.  If  a  steady  state 
core  exists,  then  the  dynamics  of  the  local  core  observers,  and  the  structure 
of  the  decision  rules,  become  stationary. 

Figure  12  illustrates  the  resulting  solution  structure.  Note  the 
decomposition  of.  each  agent's  processing  into  an  estimator  with  memory  proceed¬ 
ing  a  memoryless  decision  rule.  The  estimators  may  be  designed  independently 
of  the  cost  function  J,  but  must  be  derived  jointly.  Similarly,  the  decision 
rules  are  derived  jointly,  but  only  in  response  to  those  dynamics  retained 
in  the  core. 

D.  Special  Cases 

Three  special  cases  of  information  relations  were  identified  in  the 
last  section.  Their  impact  on  the  estimation  rules  is  summarized  here. 

1.  Splitting  Cores:  If  R*  splits,  then  the  design  of  the  decision 

1  ••  * 

rules  for  each  piece  can  be  done  separately.  If  Z^,  Z^  ,  Z^,  and 
z"  are  the  partitions  on  Z^  and  Z 2  which  support  the  split, 
and  R  and  R  are  the  restrictions  of  R  to  the  appropriate  sub¬ 
sets  ,  then 

J*(R)  =  max  {j*(r'),  J*(r"  )}  (5.19) 

and  the  optimization  over  R  decomposes. 

* 

2.  Centralized  information:  when  y^(t)  =  y2(t),  R  becomes  diagonal 
as  R(y^»  y2)  =  0  unless  y^  =  y^.  This  is  an  extreme  case  of 
spliting  as  the  value  of  the  decision  rule  for  each  case  of 


z^  *  z2  can  be  derived  separately. 


System  £ 

- 1 

Figure  12.  Optimal  Decentralized  Estimation  Structure 
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Included  information:  When  y^t)  =  h12(y2(t)),  so  agent  2  knows 
agent  l's  information  exactly,  then  R*  has  a  structure  where, 
for  every  z 2»  there  is  exactly  one  for  which  RMz^z^  f  0. 

Again  this  is  a  case  of  splitting;  decision  rules  can  be  found  via: 


a)  For  each  z ^  and  u^,  find 


Y, | z  ,u  )  =  argmin  max  J(x,u..,u  ) 

u2  xe  R(zrz2)  1  2 


(5.20) 


and  define  the  value  of  this  minimum  as  v2^zi,ui^ 


b)  For  each  z^,  set 


Y1(z1)  =  argmin  v2^zi'u1) 


(5.21) 


c)  Set 


Vz2}  =  y2u2IVyi(*i,) 


(5.22) 


This  is  possible  because  each  z  determines  the  z^  required 
in  (5.22)  uniquely. 

This  is  a  recursive  procedure  taking  advantage  of  the  nesting  of  the 
information  of  agent  1  with  that  of  agent  2.  Interpreted  as  a  Stackelberg 
solution  to  a  team  problem,  this  implies  that  agent  1,  with  the  reduced 


information,  is  a  leader  and  agent  2,  with  more  information,  is  a  followr. 
This  rather  curious  connection  between  Stack' ilberg  solutions  and  included 
observations  arises  naturally  . 
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VI.  THE  CONTROL  PROBLEM 

This  section  finally  relates  the  information  relation  and  its  properties 
to  the  full  control  problem  defined  in  section  II.  Additional  complexity 
is  generated  by  the  fact  that  information  relations  can  only  be  expanded 
in  the  context  of  specific  decision  rules.  This  sets  up  some  deterministic 
(unless  splitting  occurs)  core  dynamics  which  are  controlled  by  selections 
of  decision  rules.  Viewing  cores  as  states  and  decision  rules  as  inputs , 
a  dynamic  programming  approach  leads  to  the  complete  solution. 


A.  Information  Dynamics 

The  definitions  of  information  relation,  homomorphisms ,  automorphisms, 
containment,  and  cores  established  in  section  IV  carry  over  to  the  control 
problem  without  change.  The  expansion  process  is  the  only  place  where  new 
information  relations  are  generated,  so  the  influence  of  decisions  on 
dynamics  requires  a  modification  there. 


Definition:  The  expansion  of  an  information  relation  R 

rules  y.  :  Z.  U.  is  an  information  relation  R' , 
-  x  x 

R'  =  F(R,  Y2) 

1  1  Y 

R'  :  Zx  X  Z2  -*■  2A 


by  decision 
where 

(6.1) 

(6.2) 


Z.  =  Z.  x  ^  (6.3) 

I 

R  ((z^y^) » (Z2'y2> )  “  1(R(*j_»*2)  #Y1<*|>)«Y2  *2>> 

n  (6.4) 


□ 


This  is  interpreted  as  follows.  If  R  is  information  relation  at  a 
system  at  time  t,  then  F(R,y^,y2)  is  information  relation  between  two  agents 
observing  the  system  via  and  y2  while  it  autonomously  evolves  for  one  time 
space  step  under  the  decisions  generated  by  y^^  and  y2*  (6.4)  is  the  update 
for  the  conditional  state  set  given  R{z^,z^)  as  the  prior  conditional  state 
set,  decisions  generated  by  y^  and  y2  and  new  observations  y  and  y2« 

Thus  R  will  describe  the  information  relation  between  the  agents  if  y 
and  y2  are  implemented  as  decision  rules. 

Only  one  new  concept  is  needed. 

I 

Definition:  Let  if  be  a  homomorphism  from  R  to  R  .  Then  a  decision  rule 

■  •  « 

y^  :  Z.  ->  U.  is  contained  in  y.  =  2.  -*  U.  ,  denoted  y  v  . 
ill - 'i  11  'i  —  'i 

if 

I 

Yi<v  =  wv*  v  zi  e  zi  (6-5) 

o 

With  this  notion,  the  results  of  section  IV  generalize  to  the  control 
case  as: 

•  • 

Theorem  4:  Let  R  and  R  be  information  relations,  and  y^,  y^  decision 

rules  of  the  appropriate  structure. 

a)  If  R  is  a  primitive  information  relation,  then  so  is 
F(R,  y  ,  Y2) 

•  •  I 

b)  If  R  CR  i  YjCYj*  and  y2^-y2'  by  the  same 


homomorphism,  then 
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c)  If  R  is  a  primitive  information  relation,  then 


core (F (R,Y1,Y2>)  =  core {F (core (r) ,y^,y2) ) 


(6.7) 


U 


Proofs:  All  proofs  are  direct  extensjons  of 


a)  Lemma  5 

b)  Lemma  6 

c)  Theorem  2,  using  (a)  and  (b) . 


As  in  the  autonomous  case,  (c)  is  the  critical  fact.  The  core  information 
relations  cam  be  viewed  as  the  state  of  a  new  process  with  dynamics  (6.7) 
and  control  (Yj_»Y2)-  (*f  (6.7)  produces  a  new  core  which  splits,  then  the 

dynamics  must  be  viewed  as  nondeterministic,  as  different  sample  paths  in 
the  original  process  may  lie  in  either  of  the  two  pieces) .  These  dynamics 
establish  half  of  a  new  problem  structure. 


B.  Costs 


The  other  half  involves  the  objective.  Properties  developed  in  Section  V 
for  the  estimation  problem  also  generalize  to  the  control  problem. 


Definition:  The  cost  of  an  information  relation  R  with  compatible  decision 


rules  Yj^  Y2  is 


uVY2>  = 


J(x,y1(z1) ,  y2{z2)) 


(6.8) 


xe  R(z, ,z„) 
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The  overall  problem  objective  (2.13)  can  be  written  as 


J  ( I*  .  rj  —  max 

te(l,...,T} 


max 
x  (t) ' 
t 
yi 
t 


j(x(t)  ,Y1(y^)  ,Y2  (y^) ) 
possible  (6.9) 


Given  and  ,  the  only  x(t)'s  possible  are  given  by  R(t)  (y?,yt)  ,  the 
12  12 

primitive  information  relation  for  time  t  constructed  recursively 
from  (6.4)  (Theorem  4a  guarantees  such  an  R(t)  is  primitive).  (6.9)  then 
becomes 


a  ( r  ,  r  )  *  max  max  J  (x  (t)  ,y (yS  ,y0  (y,) ) 

1  te{if . . ,t}  t  1122 

yi  (6.10) 

t 

y2 

t  t 

xe  R(yx,y2) 

=  max  J (R (t) ,  y  (t) ,  y  (t) )  (6.11) 

te  (l,...,T) 

where  the  dependence  of  R(t)  on  prior  decision  rules  is  left  implicit. 

Thus  the  overall  objective  can  be  written  in  terms  of  costs  of  primitive 

relations  with  decision  rules. 

The  final  result  needed  is: 

•  >  « 
Theorem  5:  Let  R  and  R  be  information  re  ations,  with  y^»  Y2  and  y2 

compatible  decision  rules.  Then 

'  •  , 

a)  If  RC  R  »  YiC  *  and  Y2  CY2  by  the  same  homomorphism  <J>, 
then 


J(R,  Yj *  Y2)  1  J(R  ,  Yxr  Y2) 


(6.12) 


homomorphism  then 


-m/Y^  Y2)  =  J(R  »  Yx»  Y2>  (6.13) 

Proof;  Also  direct  extensions  of  previous  results. 

a)  Lemma  8 

b)  (a)  with  theorem  1.  ^ 

Applying  (6.13)  to  (6.11),  the  objective  becomes 

J(T  ,  T  )  =  max  J  (core  (R(t) )  ,yn  (t) ,  y_(t))  (6.14) 

1  2  te{l . T)  1  2 

•  • 

where  y^  and  y2  are  the  restrictions  of  y  and  y2  to  core  (R(t)): 

C.  General  Solution 

Let  R* (t)  =  core  (R(t)),  where  R(t)  is  the  primitive  information  relation 
created  by  decision  rules  prior  to  time  t.  From  theorem  4c, 

RMt+1)  =  core  (F(R*(t),  Yj(t),  Y2(t)))  (6.15) 

=  F*(R*(t),  Y1(t),  Y2(t)) 

and  the  overall  objective  is 

J(I\,  I\)  =  max  J(RMt),  y,  (t) ,  v,(t);  (6.16) 

1  2  tefi . t)  1  2 

where  r  and  T_  are  now  sequences  of  decision  rules  whose  arguments  are  in 
Z^t)  and  Z^  (t)  ,  respectively.  Solution  of  this  problem  would  involve 
straightforward  minimax  dynamic  programming  if  the  s<.t  in  which  the  R*(t) 
could  lie  were  determinable  ahead  of  time.  Unfortunately,  it  is  not  known 
how  to  do  this  at  this  point;  however,  one  may  construct  the  set  of  all  cores 
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reachable  under  all  from  R* (0)  and  take  this  to  be  the  requisite  set. 

This  generates  the  following  procedure  for  solving  the  decentralized 
optimal  control  problem  for  finite  nondeterministic  automata: 

1.  Using  (6.15)  for  all  possible  :  Z^(t)  ,  construct 

the  set  It  of  reachable  cores. 

2.  Dynamic  program: 

a)  Set 

V(R,T)  =  min  J(R,  Y2*  (6.17) 

Yl'  Y2 

for  each  RS  E  . 

b)  Given  V(R,t+l'  for  every  R  e  R  ,  compute 

V(R,t)  =  min  max  {  J(R,  y^,  y2)  , 

Y1'Y2  (6.18) 

V(F*  (R,  y  ,  y 2>  ,  t+1)} 

(if  F* (R,  y  ,  Y  2)  splits,  include  all  pieces  separately  in  (6.18))- 

c)  After  the  recursion  (b)  ends,  determine  the  actual  cores  R* (t) 

★  * 

and  optimal  decision  rules  y  (t),y2(t)  reached  from  R*  (0)  , 

*  * 

using  (6.15)  and  deriving  y  ,  as  the  arguments  achieving 

a  minimum  in  (6.18) 

3.  Implement: 

a)  Construct  the  local  observers  from  the  dynamics  of  Z^(t) 
and  Z2 (t)  along  the  optima)  trajectory  of  cores, 

*  * 

b)  Implement  y^tt)  and  Y2(t)  as  memory  less  feedback  maps  from 
Z, (t)  and  Z,  ft)  to  decision  sets. 

1  4 

0 


There  are  several  structural  points  to  note  about  this  solution. 

First,  the  implementation  is  of  the  same  structure  as  that  of  figure  12, 

except  both  decisions  are  now  fed  back  into  the  system.  Thus  each  aaent  consisti 
of  an  estimator  followed  by  a  memoryless  decision  rule.  As  before,  the 
local  estimators  must  be  designed  jointly;  however,  their  structure  is 
highly  dependent  on  the  decision  rules  of  both  agents .  There  is  no 
separation  principle  in  this  class  of  decentralized  control  problems: 
the  structure  of  the  estimator  for  agent  1  depends  on  the  decision  rules 
which  agent  2  uses,  as  knowledge  of  these  impacts  on  how  agent  one  "second 
guesses"  the  activities  of  agent  2  and  their  impact  on  the  system.  Finally, 
the  least  well  understood  step  is  step  1  -  for  many  problems  R_  may  be 
reasonably  small;  however,  problems  for  which  one  set  of  strategies 
generates  dynamics  such  as  those  of  figure  3c,  increasing  over  time 
without  limit,  may  yield  unmanageably  large  sets  of  R  after  a  few  steps. 

The  importance  of  this  issue  is  thus  highly  dependent  upon  the  problem. 

As  a  final  note, it  is  easy  to  verify  by  induction  that:  If  R  C  R1 , 

then 

V(R,t)  <  V(R’,t)  Vt  6  {1 . T} 

VII.  CONCLUSIONS 

Having  achieved  the  solution  of  the  general  control  problem,  several 
lessons  can  be  learned  from  this  exercise. 

A.  Summary 

The  three  basic  constructs  introduced  were  the  information  relation, 
the  notation  of  containment  of  one  relation  in  another,  and  the  process 


of  reducing  one  into  another.  These  are  the  decentralized  generalizations 


of  the  constructs  required  for  a  centralized  treatment  of  this  problem: 
sets,  set  containment,  and  the  elimination  of  redundant  elements  in  set 
unions  (in  the  computation  of  the  conditional  state  set) •  The  dynamics 
of  the  information  relation  were  derived.  The  facts  that  the  containment 
relation  is  preserved  by  these  dynamics,  and  that  it  leads  to  a  compatible 
inequality  on  the  cost  of  an  information  relation,  are  what  establish  the 
role  of  irreducible  information  relations  (the  core)  in  the  problem. 
Reformulation  of  the  control  problem  in  terms  of  the  core  yields  a 
standard  dynamic  programming  problem. 

The  core  exists  as  a  simple  description  of  one  agent's  information 
set  to  both  the  system  state  and  to  other  agents'  information.  The  fact 
that  it  is  simpler  than  the  primitive  information  relation  is  strongly 
dependent  upon  the  nondeterministic  system  model  and  the  minimax  cost 
objective.  This  is  most  evident  in  the  decentralized  estimation  case. 

At  each  point,  the  static  problem  which  each  agent  must  solve  is  originally 
posed  in  terms  of  a  primitive  information  relation,  then  simplified  to 
one  on  the  core.  Consider  the  relation  of  figure  13.  z^  can  be  mapped  onto 
Z11  by  a  re<*ucing  automorphism.  This  implies  that  we  may  set 
Y ]^Z2i^  =  yl^zll^  without  loss  of  optimality.  Let  Y2  he  any  decision  rule 
on  (z2]/z22^  *  T^e  ai5°ve  equality  assures  that  the  event  (x«l,z  ,z  ) 
will  be  penalized  identically  as  would  (x=l,z^2 ,z2^) ,  and  this  cannot 
increase  the  value  of  the  worst  case.  However,  it  may  well  be  that 

min  max  J (x,Y  (z  )  ,Y  (z  ) )  >  min  max  J  (x,Y  (z_ ,.)  ,Yo  (z. , ) 

Yl(zn)  xe(i,2}  1  11  2  21  Yl(z12)  xe{l}  1  12  2  11 

(7.1) 

These  two  could  not  be  treated  as  equal  with  other  objective  structures, 


and  thus  one  could  not  consider  them  equivalent  and  the  information 
relation  could  not  be  reduced. 

Computationally,  the  information  relation  simplifies  some  problems, 
often  significantly.  However,  two  difficulties  remains  the  core  may 
always  grow  in  size  without  bound,  and  the  number  of  possible  decision  rules 
which  may  be  defined  on  a  large  information  relation,  and  which  defines 
the  size  of  the  control  space  in  the  reformulated  problem,  may  be  quite 
large.  These  difficulties  mav  be  resolved  through  deeper  understanding 
of  the  role  of  the  state  core  or  through  branch  and  bound  techniques, 
but  these  are  open  questions 

B.  Conclusions 

Whether  these  results  may  suggest  important  new  approaches  to 
practical  decentralized  control  design,  or  whether  they  exist  only  as 
an  interesting  special  case,  is  only  a  matter  of  speculation  now. 

However,  they  allow  two  principle  conclusions  to  be  drawn. 

First,  the  information  relation  is  a  richer  structure  than  a  set,  and 
essential  to  this  problem.  Extension  of  these  results  to  probabilistic 
models, with  which  we  are  more  familiar,  will  require  a  similar  generalization 
of  probabilistic  structures  to  the  decentralized  case.  However,  behind 
every  probability  measure  lies  a  0 -algebra,  so  part  of  the  work  may  be 
done. 

Second,  there  are  some  problems  which  can  be  cast  in  the  modfil  and 
objective  structure  assumed  here,  and  they  can  now  be  addressed  directly. 
Events  which  can  be  expected  to  happen  sometimes,  but  not  always,  can  be 
modeled  by  nondeterministic  structures  which  allow  them  to  occur,  say, 


at  least  three,  but  no  more  than  five,  times  out  of  any  eight  sequential 
time  steps.  By  this  type  of  remodeling,  the  apparent  limitations  of 
the  formulation  can  be  partially  overcome . 

For  example,  consider  the  problem  of  designing  protocols  for  de¬ 
centralized  management  of  a  multiaccess  channel  [18].  A  source  of  messages 

may  produce  them  no  more  than  N  out  of  any  N  consecutive  time  steps. 

M  T 

They  may  appear  at  any  one  of  M  transmitting  sites.  There  they  are 
stored  in  a  queue,  and  the  agent  at  each  site  must  decide  whether  to 
attempt  to  transmit  one  message  or  to  wait,  at  each  time  step.  It  has 
knowledge  of  its  queue  length  and  past  activity  on  the  channel.  Depending 
on  the  number  of  agents  transmitting  simultaniously,  the  channel  may 
be  clear,  busy  (one  attempt,  successful)  or  jammed  (several  attempts, 
unsuccessful) .  A  successful  attempt  reduces  a  queue  length  by  one.  The 
objective  function  penalizes  the  maximum  length  of  the  queue  at  any  site, 
at  any  time,  over  all  possible  message  production  patterns.  This  suggests 
a  relatively  simple  nondetexministic,  finite  state  model  with  a  minimax 
objective  -  and  is  thus  suitable  for  solution  by  the  techniques  given 
here.  Variation  of  the  structure  of  the  solution  as  N  ,  N ,  and  M 

N  T 

are  changed  would  be  quite  interesting. 
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APPENDIX  A 


Steady  State  Cores:  'tonexistance 


This  appendix  demonstrates  a  system  which  does  not  have  a  steady 
state  core  for  a  decentralized  estimation  problem.  The  dynamics  and 
observation  structure  are  those  of  figure  3c,  so  the  primitive  information 
relations  are  themselves  irreducible  (figure  8) .  Let  the  decision  sets  be 
■  (J^  *  [0,1].  Construct  a  cost  functional 


J(VU2>  - 


^(U!'U2) 


x  6 {1,2,3} 


*(VlU2  -  Cl»od  1>  *  -  4 


(A-l) 


where  T(x,y)  =  0  iff  x  *  y,  and  where  e  is  any  irrational  number. 

First,  we  can  construct  decision  rules  for  this  problem  which  achieve 
a  worst-case  cost  of  0.  Clearly  no  rules  exist  which  give  better  worst-case 
performance.  Consider  any  information  relation  of  the  form  illustrated  in 

I 

Figure  8,  say  8(3).  If  z^  is  the  local  observer  state  of  agent  1 
corresponding  to  the  uppermost  row,  set  y^U^  =  ot  6[0,1]  arbitrarily. 

Now  proceed  recursively:  assume  some  cell  has  one  decision  rule  corresponding 
to  it  already  defined  (i.e.  for  the  cell  at  (z^,z2>,  either  or 

Y2(z2)  is  specified).  Construct  the  other  by  setting 


W 


y 2 (z2) 


(Y  (z  )  -  el 

mod  1 


if  r(z1,z2>  e  {1,2,3} 


(A-2) 


if  R(z1,z2)  =  4 


if  defined,  and 


1<21> 


if  R(z1*z2)  6  {1,2,3} 


^2  ^Z2^  — ' 


jY,(z ,)  +  el 

mod  1 


if  R(z1«z2)  =  4 


(A-3) 


if  it  is  Y^lz^)  that  is  defined.  This  introduces  a  new  cell  with  only  one 

component  defined,  or  terminates  the  recursion.  Note  that  aside  from  the  choice 

* 

of  a,  these  decision  rules  are  unique  in  achieving  J  =0.  Note  also 
that  the  values  of  the  decision  rules  are  of  the  general  form 


Y. (z. )  =  la  ±  n  e] 

1  mod  1 


n  integer 


(A-4) 


Now  assume  that  a  reducing  homomorphism  ♦  exists  for  an  information 

relation  of  this  type  which  produces  a  (strictly  simplified)  core.  Then 

•  • 

there  must  be  a  pair  (z.,z.)  such  that  z.  ?  z .  =  $.(z.)j  otherwise  $ 

i  i  1111 

would  be  an  isomorphism.  By  theorem  3,  the  value  of  R(3)  must  equal  that 
of  its  reduction.  From  above,  that  value  is  zero.  Also  from  above,  the 
decision  rules  which  achieve  that  value  must  be  of  the  form 


Y.  (z  )  *  la  ±  n  e] 

1  i  1  mod  1 


I  t 

Y. (z. )  *  (a  ±  n.el 
11  1  mod  1 


(A-5) 


t 

Moreover,  since  z^  +  z^,  the  recursive  procedure  for  generating  these  rules 


must  have  at  least  one  step  of  the  form 
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Yi(*)  =  [y.  (•)  ±  el  (a-6) 

3  2  mod  1 

I  I 

between  the  derivation  of  Y^zJ  and  y^zj.  Thus  ni  j*  n  .  From  the 
proof  of  Lemma  8,  we  must  be  able  to  set 

Yilzi)  =  VV  (A-7) 

and  preserve  optimality.  This  occurs  iff 

I 

[ (n  -  n  )£]  =0  (A-8) 

mod  1 

i.e.  iff  the  quantity  in  brackets  is  an  integer.  Since  e  may  be  irrational, 
we  have  a  contradiction.  All  primitive  information  relations  for  the 
system  of  Figure  3c  must  be  irreducible;  hence  the  core  of  the  system  is 
unbounded  in  size;  hence  no  finite  steady  state  core  exists. 


